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Abstract. While the standard approach to quantum systems studies length pre- 
serving linear transformations of wave functions, the Markov picture focuses on 
trace preserving operators on the space of Hermitian (self-adjoint) matrices. The 
Markov approach extends the standard one and provides a refined analysis of 
measurements and quantum Markov chains. In particular, Bell's inequality be- 
comes structurally clear. It turns out that hidden state models are natural in the 
Markov context. In particular, a violation of Bell's inequality is seen to be com- 
patible with the existence of hidden states. The Markov model moreover clarifies 
the role of the "negative probabilities" in Feynman's analysis of the EPR paradox. 

Keywords. Bell's inequality, EPR paradox, hidden state, Markovian operator, 
negative probability, quantum Markov chain, quantum measurement 

1 Introduction 

Classical models of physical systems allow a clear distinction between the current state 
of the system and the probability to extract particular information in that state. The dis- 
tinction becomes less clear in quantum models where the state of a system is given in 
terms of its probabilistic description via a quantum density, i.e., a self-adjoint matrix 
with trace 1 and nonnegative eigenvalues. This situation leads to counter-intuitive dif- 
ficulties that have been pointed out by Einstein, Podolsky and Rosen ||8l. While Feyn- 
man ifTOl introduced the idea of "negative probabilities" to resolve the EPR paradox 
theoretically. Bell |5 6 1 offered an inequality as a test against the objection of Einstein 
et ai. The violation of Bell's inequality as observed experimentally by Aspect et al. ^ 
seems to confirm the paradoxical nature of the quantum world indeed. 

We propose here a model that makes it very natural to distinguish between states 
and observation probabilities in quantum models. The mathematical key in our analysis 
is the study of general trace preserving linear operators on the space of self-adjoint 



2 U. Faigle/A. Schonhuth 



matrices rather than length preserving linear operators on the space of wave functions. 
This approach is very much in line with the classical Markov model for evolution of 
states in time and thus yields a comprehensive model for quantum Markov chains as 
a byproduct, which includes the random walk model of Aharonov et al. 1.1. 16J as a 
special case, for example (Section|4]l. Without going here into details, we mention that 
our approach also offers an alternative to Temme et al. flT\ for quantum MetropoUs 
sampling. 

The elementary units in our model are Markov densities, i.e., self-adjoint matrices 
with trace 1. Markov densities do not necessarily correspond to probabilities but are 
primarily mathematical descriptions of states of preparation of a system. It follows that a 
quantum measurement is not necessarily observable in a (macro-)statistical sense if the 
system is in a general Markov state. Hence it is meaningful to ask when a measurement 
is observable (Section[3]l. Moreover, the Markov model yields a refined tool to analyze 
notions of joint observability of not just two but three or more measurements. We show 
in Section 15.41 that the Markov perspective provides a novel look at Bell's inequality. 
In particular, it shows that a violation of Bell's inequality is not contradictory to but 
compatible with hidden state models. In fact, we argue in Section |6] that hidden state 
models are natural within (classical or quantum) Markov models of measurements. 

As was pointed out previously (see ifTSl ). the EPR paradox can be reframed as re- 
sulting from EPR's stipulation that all observables associated with the preparation of 
a system follow a joint probability distribution which reflects a probability distribution 
over the inherent hidden states. Feynman suggested to resolve the problem by allow- 
ing joint probabilities to be negative, which triggers philosophical difficulties with the 
interpretation of such quantities. Here, we allow systems to be prepared with a clear 
view on their hidden states without requiring a preparation to be probabilistic. Instead, 
we insist that meaningful observations be statistically analyzable. So, according to the 
standard tenet that quantum measurements are associated with probability distributions, 
hidden states can perhaps not be measured statistically meaningfully. But they can very 
well be meaningfully encoded in terms of (Markov) densities. 

Motivated by Wigner's ll22l distribution, akeady Dirac |7| and Bartlett f4l had sug- 
gested "negative probabilities" to arrive at appropriate mathematical models for seem- 
ingly paradoxical quantum world situations (see also Section |53] l and new probabilistic 
models have been developed for that purpose (see, e.g., Khrennikov [ 17|). The Markov 
approach clarifies the issue: Markov densities describe preparation states and not proba- 
bility distributions. While those state descriptions may involve negative numbers, there 
are no negative probabilities. All the probability distributions that are associated with 
observables in the Markov model are classical (nonnegative). There is no need for new 
probability theories. 

2 Preliminaries 

R denotes the scalar field of real numbers and C the field of complex numbers z — a+ih 
(where a,6 G M and i^ — —1). C™^" is the (mn)-dimensional vector space of all 
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(m X n)-matrices of the form 

C = A + iB with A,B e 



(1) 



C = A — iB is the conjugate of C = ^4 + iB. The transpose C* — C of its conjugate 
C is the adjoint of C. C™^" is a Hilbert space with respect to the Hermitian inner 
product 

m n 

{C\D) :=ti-(C*D)=^^c-rf,„ 

1=1 j=i 



where the Cij and the dij denote the coefficients of C and D. \\C\\ := a/ (C|C) is the 

and can also be identified with the space of diagonal 



norm of C. 
matrices in ' 



I!" is short for C" 



e C" 



diag(wi,U2, . . . 



?;i 
t;2 





Assuming m = n, a matrix C — A + iB with the property C* = C is self-adjoint 
or Hermitian, which means that A is symmetric (i.e., A^ — A) and B skew-symmetric 
(i.e., B^ — —B). Let Tin denote the collection of all Hermitian [n x n)-matrices. 
From the general form ([T]i one recognizes T-Ln as a real Hilbert space of dimension 
dimR(H„) = 11?. 

A matrix Q = [qij] S T-Ln has real eigenvalues Ai and a corresponding orthonormal 
set {vi, . . . , Vn} of eigenvectors Vi G C", yielding the spectral decomposition 



and hence trace ti{Q) = 



i=l 



(2) 



Q e T-Ln is said to be nonnegative if all eigenvalues of Q are nonnegative, which is 
equivalent to the property 

v*Qv>Q holds for all w e C". (3) 

A vector v G C" gives rise to a nonnegative element vv* G Hn and one has 

{v\v) = tr(w'i;*). (4) 

In the case tr(OT*) = 1, the matrix vv* is thought to represent a pure state of an n- 
dimensional quantum system. 

A matrix U S C"X" is unitary if the identity matrix / factors into / = UU*, i.e., 
if the row (or column) vectors of U form an orthonormal basis for C". So also U* 
is unitary. For example, an orthonormal basis {vi, . . . , u„} of eigenvectors relative to 
Q e T-Ln gives rise to a unitary matrix U* with columns vi. Where Ai, . . . , A„ are the 
corresponding eigenvalues, the linear operator x i— > Qx on C" is described with respect 
to the basis U* via the transformed matrix 



C/QC/* =diag(Ai,...,A„) GH,; 



(5) 
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3 Measurements 

Recall that a quantum measurement with scale is a finite collection X = {Ma \ 
a e S} of matrices Ma G C"^" such that the self-adjoint mati'ices Xa = MaM* 
yield a POVM (= positive operator valued measurement, (cf. M3I19I )). i.e., the Xa are 
non-negative and sum up to the identity: 

Assume that the measurement X is applied to a system 6 that is in a state of prepa- 
ration which is described by some Q G H„. If Q is a quantum density, then the param- 
eters 

PQ{a) = {Xa\Q) =ir{MaQM*a) 

are nonnegative real numbers with sum ^^^^pg (a) — tr(Q) = 1. In this case, pQ(a) 
is interpreted as the probability for the measurement X to produce the value a & S, 
which means that X behaves like a random variable that can be analyzed statistically. 
Note, however, that Q does not necessarily have to be a quantum density for X be a 
random variable in the sense above. It suffices to have tr((5) — 1 and tr(X'^°^(5) > 
for all a e S. 

Referring to an element Q G T-Ln with \i{Q) = 1 as a Markov density, we therefore 
say that X is (statistically) observable relative to the Markov density Q if we have 

tr(X('')Q) > Va G Z". (6) 

The notion of Markov densities Q thus extends and refines the measurement model: 
The system & might be in a (Markov) state of preparation that does not allow X to be 
analyzed statistically. If Q is nonnegative (and hence a quantum density), X is neces- 
sarily observable in Q. 

3.1 Markov operators 

A Markov operator (of Markovian) is a linear map ji : Hn T~Ln that maps Markov 
densities onto Markov densities. In other words: The Markovian is a trace preserving 
linear operator on H„. /i is said to be nonnegative if it maps quantum densities onto 
quantum densities (and hence nonnegative matrices onto nonnegative matrices). 

Remark 1 In the case of complete positivity (i.e., (/ ® /i) [A) > O for any nonnegative 
A G W (8) V, where W is an extra system ) a Markovian operator is a quantum operation 
(cf. 4791/ ). The quantum operations formalism aims at modeling the dynamics of open 
quantum systems and quantum noise, thereby also borrowing from the interrelation be- 
tween classical noise and classical Markov chains. Time-discrete quantum Markovian 
dynamics have also been described by trace-preserving quantum operations as quan- 
tum channels (see, e.g., i l23l/ ). The more general Markovian formalism allows us to 
interpret quantum systems in terms of hidden states (see Section\5} and to summarize 
ergodic properties conveniently. 
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Consider, for example, any quantum measurement X = {Ma \ a £ S}. The 
associated sum operator 

Q^^lx{Q) ■.= J2MaQM: 

is nonnegative and trace preserving and hence nonnegative Markovian. In the special 
case of a measurement Xjj — {[/} with "unary scale" E (i.e., \E\ = 1), [/ is a unitary 
matrix with associated Markov operator 

Q^fiuiQ) .^UQU*. 

Any trace preserving operator ^ : V ^ V on some subspace V C "H^ extends to 
a trace preserving operator (and hence Markovian) Ji : Hn — > in the obvious way. 
Consider, for example, the space V of diagonal matrices D G Hn- A trace preserving 
operator fi : V V is described by a 

/x(diag(x)) = diag(Af (x)) {x e M"), 

where M G R"^" is a Markov matrix (i.e., M has column sums 1). /i is nonnegative 
on V if and only if all coefficients of M are nonnegative scalars, in which case M is 
the transition matrix of a (classical) homogeneous Markov process. 

3.2 Markov measurements 

The quantum measurement formalism generalizes naturally. We call a family X = 
I a G S} of linear operators fia '■ H„ — ^ Hn a Markov measurement with (finite) 
scale S if 

Ma is a Markov operator. 

X is (statistically) observable relative to the Markov density Q if 
tr(^a((3)) > holds for all aeS. 

Again, observability means that the measurement X behaves like a random process that 
produces the output a G S with probabihty (a) = tx{p.a{Q))- 

Remark 2 When the are completely positive, the Markov measurement X is a mea- 
surement model in the sense of the quantum operations formalism (see M9'i ). 

The application of an observable X moves the system under investigation with prob- 
ability pq (a) from Q into a state of preparation that is described by the Markov density 
Qa such that 



PQ{a)Qa = MaQM* and hence pqia) = ti{MaQM*). 
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So the expected Markov density after the apphcation of X is 

5IPQ(a)Qa-A^x(Q). (7) 

Muhinomial expansion of the Markov measurement X ~ {/i'^"^ | a G S} yields 

ai ,. . . ,at (El^ w^S^ 

and exhibits X* := {^^ | w G Z'*} as a Markovian measurement with scale for any 
t = 1,2, . . .. We say that X is t-observable in Q if X* is observable in Q. Obviously, 
X is always 0-observable. 

Lemma 1. AssMme f/zaf f/ie Markov measurement X is t-observable in Q. Then X is 
k-observable in Q for all k < t — 1. 

Proof. It suffices to consider k = t ~ 1. Let w — ai . . .at-i G S*^^ he arbitrary. 
Since X* is observable and Markovian, one has 

HfJ-wiQ)) = tT{flxif^^viQ))) = "'(A*«.a(Q)) > 0. 

o 

If X is fc-observable in Q, the application of X'' yields the expected Markov density 
li\{Q). The following is straightforward. 

Lemma 2. Assume that the Markov measurement X is t-observable in Q. Then X is 
observable in ^^{Q) for all k < t — \. 

o 

We say that X is completely observable in Q if X is t-observable in Q for alH > 1. 

Remark 3 With the methods of [W] one can show that completely observable Markov 
measurements X yield asymptotically mean stationary processes. Such processes are 
known to liave good ergodic properties (see M3\12'i ). For example, tlie conditional en- 
tropies converge to a limit 

Hoo{X) = lim -H{X*) = lim H{X*\X*-^). 

In the case of diagonal matrices, complete observability is equivalent to the much stud- 
ied concept of finite-dimensional ( classical ) stochastic processes or observable opera- 
tor models that provide a proper generalization of classical (possibly hidden) discrete 
Markov chains with finite alphabets (see l\l 1114115]! }. We are lead to the latter model in 
more detail in Section^ 
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4 Markov chains 

Let /i be a Markov operator on Hn- Then any Markov density P gives rise to a chain of 
Markov densities n* {P), 



where is the identity operator on Hn- We refer to /i as the evolution operator of the 
Markov chain (/i, P). In the case = P, the chain (/i, P) is stationary. Note that 

classical Markov chains correspond to Markov chains of nonnegative diagonal matrices 
in the general model. 

Remark 4 The Markovian evolution operator p, does not have any density or proba- 
bility interpretation in its own right but simply describes how the densities in a chain 
{p, P) evolve over time. 

The Markov chain (/i, P) is bounded if there exists some c G K such that 



Assume, for example, that p in nonnegative and P a quantum density. Then the Markov 
chain {ji, P) is bounded because each /i* (P) is a quantum density and therefore satisfies 



Evolution of wave functions. Any unitary matrix U G C"X" induces a nonnegative 
Markovian pu{C) — UCU*. On the other hand, a wave function v G C" of norm 
\\v\\ = 1 is transformed under U into the wave function Uv of norm \\Uv\\ = 1, which 
corresponds to the pure quantum state 



So {pu, '>^v*) is the Markov chain of the pure quantum states that correspond to the 
wave function evolution v ^ U*'v for t — 0,1,.... 

Quantum random walks on graphs. A model for a quantum random walk on a di- 
rected regular graph G has been proposed by Aharonov et al. lH] (see also Kempe [[T6l ). 
A natural generalization of that model to arbitrary directed graphs on V is the following. 

Given the graph G — {V,£) with node set V and edge ssl £ QV xV, we identify 
the elements e G f with the elements in the standard basis of unit vectors in C". A 
quantum walk on G is now described in terms of a Markovian operator p : Hn Hn- 
The walk on G starts at time t = in a quantum density state P^^^ and consists in 
subsequent density changes according to 



The quantum walk corresponds to the Markov chain (/x, P'*'^) in our terminology. Let 
us define for every node v & V the diagonal matrix Xy = diag(x'^"^) G ?^„, where the 
vector x*^"-* G C" has the components 



{p,P) ■.= {p\P)\t = Q,l,...}, 



{p\P)\p\P)) = tr(/i*(P)2) < c holds for all t. 



ir{p\Pf) < I. 



{Uv){Uv)* = Uvv*U* = nu{vv*). 



p(*) := /i(P^*"^^) for all t 1, 2, . . .. 




1 if e G £ is of the form e — {v, w) 
otherwise. 
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Note that the family X = {Xy \ v e V} constitutes a POVM. If all P'^*^ are nonnega- 
tive, the apphcation of the matrices X'-'^'^ yields the nonnegative parameters 

Pi(«) = (X„|p(*))=tr(X„p(*))>0, 

which means that X is observable. pt{v) is then interpreted as the probability for the 
random walk to be in node v at time t. 

Aharonov et al. [ 1 1 assume a unitary wave function evolution and hence a non- 
negative Markovian operator of the type iJ.u{P) — U PU* . Using the assumed regular- 
ity of the underlying graph G, they derive an ergodic property of their quantum walk 
and show that the following limit of averages exists in their model: 

1 

p{v) = lim - ^Pk{v) . 

t-foo t — ' 

We will exhibit a general convergence property for bounded Markov chains below. 

Remark 5 Our general quantum random walks need not respect the structure of the 
underlying graph. This is purely for formal convenience: Our model incorporates the 
most general graph-respecting concepts of quantum walks suggested in iTj. 

It is of interest to know whether a Markov chain converges towards a stationary 
behavior. We consider a fixed Markovian /i : Tin — > Hn- 

Theorem 1. Let (p, P) be a bounded Markov chain. Then the limit of averages 

P= lim (8) 

fc=l 

exists and (/i, P) is a stationary Markov chain. Moreover, if fi is nonnegative and P a 
quantum density, then also P is a quantum density. 

If the limit (|8]l exists, it is clear that tr(P) = tr(P) = 1 holds and (/i, P) is station- 
ary. Moreover, if ji preserves quantum densities, then each fJ,*{P), and therefore each 
average, is a quantum density. So it remains to prove the existence of P. It is convenient 
to base the proof on the following lemma. 

Lemma 3 (iSl). Let V be a finite-dimensional normed vector space over C and con- 
sider the linear operator F :V V. The following statements are equivalent: 

(a) V = limt_i.oo 7 X]l=i F'^i^) exists for all v V. 

(b) For every v ^ V, there exists some finite bound c G R such that ||P*(w) |j < c holds 
for all t > 0. 
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We want to apply Lemma[3]to the (complex) subspace V C C"'*" that is generated 
by ifJ.,P) with the norm ||C|| = ^tr(C*C). To this end, we augment _P to a basis 
{Qo, Qi, ■ ■ ■ , Qm} '!= (Mi P) for V with Qq ~ P and define the linear operator F : 
y -> y via 

m 771 

3=0 3=0 

Then F agrees with fi on the Markov chain (/x, P). Let ci be a bound on (/i, P) and 
observe from the triangle inequality: 

m m m 

1 1 z— 1 

So F satisfies condition (b) and hence also (a) of Lemma|3] which establishes the con- 
vergence of the averages in Theorem[T|with the choice v = P. 

Corollary 1. Let (fi, P) be a bounded Markov chain and X : T-Ln K any linear 
functional. Then 

lim - j2x{^,\p))=x{p). 

k=l 

o 

Remark 6 In contrast to density averages, wave function averages in a wave function 
evolution will generally not converge to a wave function. If U is a unitary matrix that 
does not have 1 as an eigenvalue, one has 

1 * 

lim - V C/*i; = for all v £ C". 

t=fOO t ^ ■' 
k=l 

5 Hidden states and Bell's inequality 

Let (3 be a physical system with a finite set /2 = {wi, . . . , wat} of hidden states and 
assume that & is (definitely) in one of the N possible hidden states w 6 at any 
discrete time t = 0, 1, .... It may not be possible to observe the true hidden state lu = 
w'^*'' at time t directly. So we want to collect statistical information on the true state. 

5.1 Information functions 

An information function with scale Z' is a function X : f] —i' E. Since i? is finite, 
we may assume S to be finite as well. Equivalently, X can be viewed as a POVM 
{Xa \ a ^ U} of diagonal matrices Xa = diag(a;^")), with xL"^ = 1 if X{uj) — a. 
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Similar to our terminology in Section [3] we call a vector q G MJ^ with components 
a Markov state of © if 

^ <Z^ = 1- 

For any a e Z", we set 

uj:X {uj) — a 

and say that the information function X is (statistically) observable in q if all the Pq{a) 
are nonnegative numbers. In the case of observability and a real-valued information 
function X, we obtain the well-defined expectation 

E,{X) := ^ X • p,ix) - 



5.2 Joint observations 

We say that the k information functions Xi : — > i^i , . . . , X^ : Q ^ on the 
system (5 are jointly observable in the Markov state g if the composite information 
function X : Q ^ S with 

:= (Xi(cj),...,Xfe(w)) and 17 := x . . . x iTfe 

is observable in q. The following statement is easy to verify. 



Lemma 4. Assume that the collection ofk information functions Xi, . . . , Xk is jointly 
observable in the Markov state q, then every subcollection Xi-^ , ■ • • , Xi^ is jointly ob- 
servable in q. In particular, every individual information function Xi is observable. 
Moreover, if the Xi are real-valued, also every product XiXj is observable in q. 

O 



Hence, if two information functions X and Y on the system © are real-valued and 
jointly observable, their product XY is statistically observable and has a well-defined 
expectation E{XY). 

Clearly, any collection of information functions is jointly observable in any non- 
negative Markov state q, i.e., classical probability distribution q on f2. 



5.3 Feynman's approach to the EPR paradox 

Feynman fTol has given a mathematical model to explain the Einstein, Rosen and 
Podolsky (EPR) paradox (see also Scully et al. |20J). In essence, Feynman provides 
the example of two information functions X, Z for the spin along the -\-x and +z axis 

of a spin 1/2 system © with f2 = {(++), (H — ), ( — h), ( )} as in the following 

table: 

_(++) (+-) (-+) (— ) 
X 

z 
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and the "joint probability distribution": 

P{++) = [l + {cr.) + {cr.) + {<Ty)]/^ 
P{+-) = [l + {o.)-{a,)-{ay)]lA 
P{-+) = [l + {o.) + {a^)~{cTy)]/A 

Pi—) = [1 - - {^.) - i^y)]/^, 

where {<Jx) , {(Ty) , {<Jz) are the Pauli spin operators. Noticing that those parameters 
P{xz) might take on negative values, Feynman interprets them as (possibly) negative 
probabilities and argues that probability theory should be expanded in this direction. 

In our terminology, the P{xz) yield a blueprint for attainable Markov states (de- 
pending on the actual values of the spin operators), where X and Z are not always 
guaranteed to be jointly statistically observable. For example, the situation 

(-7.) = (ay) - (a.) - 1/2, 

would result in the Markov state q = (5/8, 1/8, 3/8, —1/8), where X and Z are indi- 
vidually but not jointly observable. 



5.4 Bell's inequality 

The well-known inequality of Bell 05|6| takes the form (|9]l in our context as a statement 
on the expectations of products of pairs of information functions. 

Lemma 5 (Bell's inequality). Let X,Y,Z : Q { — 1,+1} be arbitrary information 
functions on the system (5. If X,Y and Z are jointly observable in the Markov state q, 
then the following inequality holds: 

\EgiXY) - Eq{YZ)\ < 1 - EyiXZ) . (9) 

Proof. Any choice of x,y,z G +1} satisfies the inequality \xy ~ yz\ < 1 — xz. 
Because of the joint observability assumption, all the observation probabilities 

Pqix, y, z) = Pr{X = x,Y = y,Z = z} 

are nonnegative real numbers that sum up to 1. So we conclude 

\Eq{XY) - Eq{YZ)\ = \^[xy ~yz)pq{x,y,z)\ < ^ \xy - yz\pg{x,y, z) 

x,y,z x,y,z 

< ^(l-a;z)p,(x,y,z) = l-Eq{XZ). 

O 

Bell's inequality may be violated by information functions that are pairwise but not 
jointly observable. Consider a system & with a set i7 = {wi, W2, ws, W4, ws} of five 
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hidden states, for example, and three information functions X,Y,Z : J7 — )• { — 1,+1} 
as in the following table: 







W2 


W3 LOi 


1^5 


X 


-1 


+ 1 


-1 -1 


-1 


Y 


+1 


+ 1 


-1 +1 


-1 


Z 


+1 


+ 1 


+ 1 -1 


-1 



One can check that X, Y, Z are pairwise observable in the Markov state 

g= (-1/3, 1/3, 1/3, 1/3, 1/3) 
and yield the product expectations 

Eq{XY) = +1, Eq{YZ) = -1/3, Eq{XZ) = +1 , 
which violate Bell's inequality 

Remark 7 Experimental results seem to indicate that quantum systems may violate 
Bell's inequality (see, e.g.. Aspect et al. /S^/j. This is sometimes interpreted as showing 
that quantum mechanics does not admit a theory with hidden variables. The Markovian 
picture makes it clear that a violation of Bell's inequality only shows that the system is 
studied in terms of measurements that are perhaps pairwise but not jointly observable. 
The existence of definite but hidden states is not excluded. In fact, an experimentally 
observed violation of Bell's inequality suggests that one should not place a priori non- 
negativity restrictions on concepts of states into which a system can be prepared. 

6 Measurement processes and hidden states 

Let us now argue that statistical models for the analysis of measurements involve Markov 
states with possibly negative components quite naturally. 

Suppose that some measurement X with finite scale S is made on some (not further 
specified) system 6 at discrete times i = 1, 2, . . .. Let Xt be the result observed at time 
t and assume that the results of the measurements are jointly observable in the sense 
that the Xt are random variables with joint probabilities 

p{ai . ..at) = Pr{Xi = ai, . . . ,Xt ^ aj for all ai, . . . , at e 

For any w = ai . . . at and v — at+i . . . 04+^, the conditional probability 

p{v\w) = Pr{X4+i = flt+i, . . .,Xt+s = at+s\Xi = ai, . . . = aj 

is the probability for correctly predicting the sequence v to occur in the next s observa- 
tions when w is obtained up to time t. (For formal reasons, it is convenient to assume 
that the empty symbol □ is issued at time t = with probability p{\I\) — 1.) We collect 
all the prediction probabilities into the (infinite) prediction matrix 

V — ^{v\w)\. 
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■p is a Hankel matrix. The rank rk (V) is also known as the dimension of X. For exam- 
ple, classical (possibly hidden) Markov chains with finite alphabet S can be shown to 
be finite-dimensional (fU'WlS |). Moreover, finite-dimensional processes are asymp- 
totically mean stationary (see |9|). 

In the case m — rk (V) < oo it is not difficult to see (cf. |fT5l ) that {Xt) admits an 
observable operator model in the following sense: 

• There is a family X = {Ma\a G S} of real (m x m)-matrices Ma and a 
Markov state tt e M™ such that for all ai , . . . , a* G i7 

(1) p(ai ... at) = tr(diag(M„,Afa,_i ■ • ■ M^^Tr)); 

(2) M — J2aes is a Mai-kov matrix. 

In our terminology, this means that X represents a Markov measurement with re- 
spect to diagonal matrices that is completely observable relative to the Markov density 
n = diag(7r). 

Remark 8 Finite-dimensional measurement processes that do not admit an observable 
operator model with only nonnegative Markov states are known to exist ( cf. 1 ,1511 ). 

Let {Xt) be a measurement process on a system 6 with finite alphabet S and finite 
dimension m and the associated observable operator model {{Ma\a G Z'}, tt). Consider 

^ = e = 1, . . . ,m} 

as the set of iV = |I7| • m hidden states of & with the information function 

X : S7 ^ S such that X{a,j) = a. 
With each Ma = [m\f] G M™^", we associate the {N x iV)-matrix 

Ma = [m(^b,i),{b,j)] with m(f,,i)^(cj) = 

Let furthermore W be the A^-dimensional vector with coordinates 

j) = 7r(j)/l'£'l- 

By construction, we have now for all w = ai . . . at, 

p{w) = tr(diag(M^7f)) . 

The process {Xt) can thus be interpreted as emanating from & via the information 
function X. The system is prepared to be in the Markov state 7f at time t = 0. At time 
t > 1, the system is in a hidden state lj E il yielding the information X{uj) = a with 
probability 

PY{Xt = a} = 7f(*) (w) (where 7f(*) = mV). 

uj:X {uj)—a 



i^^ if 6 = a 
if 5 7^ a. 
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